Image processing apparatus, image processing method, and program

ABSTRACT

The present invention provides an image processing apparatus including a skeleton component separation unit configured to separate a skeleton component illustrating a perspective structure from a source image, a mist/fog correction unit configured to calculate, based on color information in each color channel of the skeleton component, a correction factor that lowers brightness of a pixel as the pixel is brighter and correct, based on the correction factor, brightness of each pixel of the source image, and a brightness restoration unit configured to restore brightness of the brightness-corrected source image to brightness of environmental light as target brightness.

TECHNICAL FIELD

The present invention relates to an image processing apparatus, an imageprocessing method, and a program.

BACKGROUND ART

Under circumstances where mist/fog exists, as it is illustrated in FIG.8, reflection light from an object to be captured is attenuated due to ascattering of particles of mist/fog while routing from the object to acamera sensor. At the same time, environment light is also scattered bythe particles of mist/fog and thus scattered environment light alsoreaches the camera sensor. Therefore, the light applied to the camerasensor becomes mixed light of the attenuated reflection light from theobject and the scattered environmental light. Observation light I(x, λ)having a wavelength λ at a pixel location x is expressed by an equation(1) by using reflection light J(x, λ) and the environmental light A(λ)at the pixel location x. Here, t(x, λ) indicates transmittance of thereflection light. The transmittance of the reflection light is expressedby an equation (2) by using a diffusion coefficient k per unit distanceand a distance d to the object in a case where a condition of theenvironmental atmospheric air is uniform.

I(x,λ)=t(x,λ)·J(x,λ)+(1−t(x,λ))·A(λ)  (1)

t(x,λ)=exp(−k(λ)·d(x))  (2)

In a visible light wavelength region, the scattering of light caused byparticles of mist/fog is considered as being the same independent from awavelength of the light. Thus, the observation light I(x, λ) can beexpressed by an equation (3).

I(x,λ)=t(x)·J(x,λ)+(1−t(x))·A(λ)t(x)=exp(−k·d(x))  (3)

In a video image restoration technique under a condition where mist/fogexists, unattenuated reflection light J(x, λ) from the object isestimated from the observation light I(x, λ) to output an image. Morespecifically, by estimating a transmittance t(x) of the reflectionlight, the reflection light J(x, λ) is calculated using an equation (4).

$\begin{matrix}{{J( {x,\lambda} )} = {{\frac{1}{t(x)}{I( {x,\lambda} )}} - {\frac{1 - {t(x)}}{t(x)}{A(\lambda)}}}} & (4)\end{matrix}$

In the above described estimation and restoration, 2 pieces ofinformation such as the reflection light J(x, λ) and the transmittancet(x) are required to be estimated for each pixel from the observationlight I(x, λ), so that the equation results in being an ill-posedproblem where no solution can be found. Consequently, it is requiredthat optimum solutions of the reflection light J(x, λ) and thetransmittance t(x) should be estimated based on preliminary providedknowledge about the environment.

A certain number of techniques in which the reflection light and thetransmittance are estimated for removal of mist/fog are proposed todate. Among them, a method for performing correction processing withrespect to a piece of image as an input is described below withreference to non-patent document 1 and non-patent document 2.

The non-patent document 1 discloses a technique in which a restorationimage is generated based on a statically-obtained knowledge that anature description image without mist/fog generally includes, around atarget pixel, a pixel whose value is 0 in either one of a red (R)channel, a green (G) channel, and a blue (B) channel. Therefore, if apixel whose value is 0 does not exist around a target pixel, assumingthat such situation is a result of an effect of superimposition ofenvironmental light due to mist/fog, transmittance is calculated basedon an amount of the superimposition.

Further, the non-patent document 2 discloses a method for separatingreflection light and environmental light from each other focusing uponuncorrelation between a distance to a texture of an object and adistance to the object (i.e., a degree of superimposition ofenvironmental light due to mist).

CITATION LIST Non-Patent Literature

[Non-Patent Document 1] Kaiming He, Jian Sun, and Xiaou Tang, SingleImage Haze Removal Using Dark Channel Prior. IEEE Conference on ComputerVision and Pattern Recognition (CVPR), 2009[Non-Patent Document 2] Raanan Fattal, Single Image Dehazing. ACMSIGGRAPH 2008, 2008.

SUMMARY OF INVENTION Solution to Problem

However, the technique of the non-patent document 1 includes such aproblem to be solved that an excessive correction is made on a regionwhere the above described condition is not satisfied (e.g., regions ofsky and white buildings).

The technique of the non-patent document 2 is expected to theoreticallygenerate a high-quality image; however, as a result of a simulation,obtained in a region widely occupied by superimposition of a lightsource component due to mist was a low quality output. This is becausedifferent information between color bands is used as informationsuggesting a texture of an object. More specifically, in the regionwidely occupied by the superimposition of the light source component dueto mist, such region is susceptible to be affected by a noise because ofan extremely small differential value between color bands.

The present invention is made to provide an image processing apparatuscapable of performing high quality mist/fog correction processing evenin a case where a pixel whose value is 0 in either one of a red (R)channel, a green (G) channel, and a blue (B) channel does not existaround a target pixel and in a case where a different value betweencolor bands is small, image processing method, and a program.

Means for Solving the Problems

The present invention is directed to an image processing apparatusincluding a skeleton component separation unit configured to separate askeleton component illustrating a perspective structure from a sourceimage and a mist/fog correction unit configured to calculate acorrection factor based on color information in each color channel ofthe skeleton component and correct, based on the correction factor,brightness of each pixel of the source image.

The present invention is directed to an image processing methodincluding separation processing configured to separate a skeletoncomponent illustrating a perspective structure from a source image andmist/fog correction processing configured to calculate a correctionfactor based on color information in each color channel of the skeletoncomponent and correct, based on the correction factor, brightness ofeach pixel of the source image.

The present invention is directed to a program for causing a computer toexecute separation processing configured to separate a skeletoncomponent illustrating a perspective structure from a source image andmist/fog correction processing configured to calculate a correctionfactor based on color information in each color channel of the skeletoncomponent and correct, based on the correction factor, brightness ofeach pixel of the source image.

Advantageous Effect of Invention

According to the present invention, high quality mist/fog correctionprocessing is achieved even in a case where a pixel whose value is 0 ineither one of a red (R) channel, a green (G) channel, and a blue (B)channel does not exist around a target pixel and in a case where adifferent value between color bands is small.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an image processing apparatusaccording to a first exemplary embodiment.

FIG. 2 is a block diagram illustrating an image processing apparatusaccording to a second exemplary embodiment.

FIG. 3 is a block diagram of an image processing apparatus according toa third exemplary embodiment.

FIG. 4 is a graph illustrating a reference level.

FIG. 5 is a graph illustrating environmental light.

FIG. 6 shows a source image (i.e., an input image) and an output image.

FIG. 7 is a block diagram illustrating an image processing apparatusaccording to a fourth exemplary embodiment.

FIG. 8 illustrates a background art.

DESCRIPTION OF EMBODIMENTS

A first exemplary embodiment of the present invention is describedbelow.

FIG. 1 is a block diagram illustrating an image processing apparatusaccording to the first exemplary embodiment of the present invention.

The image processing apparatus according to the first exemplaryembodiment includes a skeleton component separation unit 1, a mist/fogcorrection unit 2, and a brightness restoration unit 3.

The skeleton component separation unit 1 is configured to separate askeleton component illustrating a perspective structure from a sourceimage (i.e., an input image) input to the image processing apparatus. Anexample of the separation method includes a skeleton component/texturecomponent separation method, using a total variation norm minimization,capable of extracting brightness and color information for each region.The total variation norm minimization is a technique for removing avibration component within an image as a texture component or a noisecomponent. More specifically, in the technique, a skeleton component ofa source image (i.e., an input image) is extracted by removing a texturecomponent or a noise component from the source image.

More specifically, by solving a minimization problem expressed by anequation (5), a skeleton component B of an image is extracted from asource image (hereinafter referred to as input image) I. In combinationwith a multi-resolution analysis, not only a removal of a fine vibrationcomponent but also a removal of a (low-frequency) vibration componenthaving a wide cycle to some extent can be achieved. Here, subscript “2”in the equation (5) indicates L2-norm.

$\begin{matrix}{\min\limits_{B}( {{\int{{{\nabla B}}{x}{y}}} - {\frac{\mu}{2}{{I - B}}_{2}^{2}}} )} & (5)\end{matrix}$

The mist/fog correction unit 2 is configured to calculate, based oncolor information in each color channel of the skeleton component, acorrection factor that lowers brightness of a pixel as the pixel becomesbrighter and correct, based on the correction factor, brightness of eachpixel of the input image and each pixel of the skeleton component toremove mist/fog. In a correction of the brightness, a gamma correctionis locally performed to suppress a drastic fluctuation on a lowbrightness side of the input image, thereby preventing a black defect.More specifically, the mist/fog correction unit 2 is configured tocalculate, from a minimum value of color channel of each pixel of theskeleton component, an amount of superimposition of mist/fog of the eachpixel and calculate a gamma value (i.e., a correction factor) forcorrection of a pixel from the amount of superimposition of mist/fog toperform a gamma correction by using thus calculated gamma value withrespect to the pixel. Incidentally, the gamma value (i.e., thecorrection factor) may be determined not only with color information ineach color channel but also additionally with brightness surrounding thepixel to be corrected.

More specifically, the mist/fog correction unit 2 is configured toperform the gamma correction, with reference to the skeleton componentB, while changing the gamma value for each pixel based on the colorinformation of each pixel of the input image I. Further specifically,with respect to a pixel at a location x, a value obtained such that aminimum value of each color channel of the skeleton component B ismultiplied by a ratio r is set to an amount of superimposition haze(0<haze<1) of mist/fog of the image (see, an equation (6)). Then, agamma value γ_(h) is set for the purpose of a correction by using anequation (7) and correction is performed based on the gamma value γ_(h)according to an equation (8). Here, T₁ indicates an input image afterremoving mist/fog, x indicates a location of a pixel, and λ indicates acolor channel.

$\begin{matrix}{{haze} = {r \cdot {\min_{\forall\lambda}( {B( {x,\lambda} )} )}}} & (6) \\{\gamma_{h} = \frac{1.0}{1.0 - {haze}}} & (7) \\{{T_{1}( {x,\lambda} )} = ( {I( {x,\lambda} )} )^{\gamma_{h}}} & (8)\end{matrix}$

The brightness restoration unit 3 is configured to restore brightnessthroughout an image whose brightness was lowered to brightness ofenvironmental light of the image as target brightness with respect tothe input image whose brightness was corrected. The mist/fog correctionunit 2 will perform correction processing around a saturation point 1.0in order to suppress information loss caused by saturation. Accordingly,brightness of the environmental light and brightness throughout theimage tend to be dark. Now, thus lowered brightness throughout the imageis restored by the brightness restoration unit 3. More specifically, thebrightness restoration unit 3 is configured to calculate an index γ₂ forconverting thus lowered brightness of the environmental light to targetbrightness and correct, by using the index γ₂, the entire imageaccording to an equation (9).

T ₂(x,λ)=1.0−(1.0−T ₁(x,λ))^(γ) ²   (9)

Here, T₂ indicates an input image whose brightness has been restored.

The index γ₂ can be calculated by the following equation:

γ₂=ln(1.0−q)/ln(1.0−p)

where a value of the environmental light after being subjected to mistcorrection processing is p and target brightness is q. Here, the value pmay be a correction result obtained in white correction processing.

According to an aspect of the first exemplary embodiment, in themist/fog correction processing for removing mist/fog from an image, ablack defect can be prevented by suppressing a drastic fluctuation onthe low brightness side of the input image. According to another aspectof the first exemplary embodiment, decrease in contrast can also beprevented.

A second exemplary embodiment of the present invention is describedbelow.

FIG. 2 is a block diagram illustrating an image processing apparatusaccording to the second exemplary embodiment of the present invention.

The image processing apparatus of the second exemplary embodimentincludes a skeleton component/texture component separation unit 11, amist/fog correction unit 12, a brightness restoration unit 13, and anoise suppression unit 14.

The skeleton component/texture component separation unit 11 isconfigured to separate a source image (i.e., an input image) input tothe image processing apparatus into a skeleton component illustrating aperspective structure and a texture component as a residual component ofthe source image. An example of the separation method includes askeleton component/texture component separation method, using a totalvariation norm minimization, capable of extracting brightness and colorinformation for each region. The total variation norm minimization is atechnique for removing a vibration component within an image as atexture component or a noise component. In the technique, the removal ofthe texture component or the noise component enables extraction of askeleton component of a source image (i.e., an input image). Morespecifically, the skeleton component B of the image is extractedaccording to the above described equation (1) and the skeleton componentB is subtracted from the source image (hereinafter referred to as inputimage) I.

The mist/fog correction unit 12 is configured to calculate, based oncolor information in each color channel of the skeleton component, acorrection factor that lowers brightness of a pixel as the pixel becomesbrighter and correct, based on the correction factor, brightness of eachpixel of the input image and each pixel of the skeleton component toremove mist/fog. In the brightness correction, execution of the localgamma correction suppresses a drastic fluctuation on a low brightnessside of the input image to prevent a black defect. More specifically,the mist/fog correction unit 12 is configured to calculate, from aminimum value of color channel of each pixel of the skeleton component,an amount of superimposition of mist/fog of the each pixel andcalculate, based on the amount of superimposition of mist/fog, a gammavalue (i.e., a correction factor) for correcting the each pixel toperform a gamma correction by using thus calculated gamma value withrespect to the each pixel. Incidentally, the gamma value (i.e., thecorrection factor) may be determined not only with color information ineach color channel but also additionally with brightness surrounding thepixel to be corrected.

More specifically, a resulting input image T₁ is almost equivalent tothe resulting input image of the mist/fog correction unit 2. Theskeleton component B (x, λ) can be obtained in a similar manner by usingequations (6)′, (7)′, and (8)′ and an application of mist/fog removal,thereby obtaining B₁(x, λ)

$\begin{matrix}{{haze} = {r \cdot {\min_{\forall\lambda}( {B( {x,\lambda} )} )}}} & (6)^{\prime} \\{\gamma_{h} = \frac{1.0}{1.0 - {haze}}} & (7)^{\prime} \\{{B_{1}( {x,\lambda} )} = ( {B( {x,\lambda} )} )^{\gamma_{h}}} & (8)^{\prime}\end{matrix}$

The brightness restoration unit 13 is configured to have a functionsimilar to the function of the above described brightness restorationunit 3. The brightness restoration unit 13 is configured to restore notonly brightness of the input image whose brightness was corrected butalso, with respect to the skeleton component, brightness throughout animage whose brightness was lowered to brightness of environmental lightof the image as target brightness. The brightness-restored input imageT₂ is obtained in a similar manner by using the equation (9) and thebrightness-restored skeleton component B₂ is expressed by an equation(9)′.

B ₂(x,λ)=1.0−(1.0−B ₁(x,λ))^(γ) ²   (9)′

The noise suppression unit 14 is configured to suppress a noise of thetexture component based on the brightness-restored input image, thebrightness-restored skeleton component, and the texture component andgenerate an output image from the noise-suppressed texture component andthe brightness-restored skeleton component. That is, the noisesuppression unit 14 is configured to suppress a sharpened noise in animage. In the mist/fog correction processing, since a local contrast issharpened, the noise as well as the texture component is sharpened inthe image. The noise suppression unit 14 is configured to suppress thesharpened noise with respect to the input image T₂ and the skeletoncomponent B₂. More specifically, a brightness enhancement ratio t oftexture and noise is calculated according to an equation (10) by usingthe skeleton component B₂ having been subjected to correction processingsimilarly in the input image.

$\begin{matrix}{{t( {x,\lambda} )} = {\frac{{T_{2}( {x,\lambda} )} - {B_{2}( {x,\lambda} )}}{{I( {x,\lambda} )} - {B( {x,\lambda} )}} = \frac{{sub}_{2}( {x,\lambda} )}{{sub}( {x,\lambda} )}}} & (10)\end{matrix}$

Now, based on the brightness enhancement ratio t, an attenuation F(t) ofthe texture component is set to generate an output image J according toan equation (11).

$\begin{matrix}{{J( {x,\lambda} )} = \{ \begin{matrix}{B_{2} + {{t( {x,\lambda} )} \cdot {{sign}( {{sub}( {x,\lambda} )} )} \cdot}} & {{{if}{{{sub}( {x,\lambda} )}}} > {F( {t( {x,\lambda} )} )}} \\( {{{{sub}( {x,\lambda} )}} - {F( {t( {x,\lambda} )} )}} ) & \; \\{B_{2}( {t( {x,\lambda} )} )} & {else}\end{matrix} } & (11)\end{matrix}$

Here, the attenuation F(t) of the texture component is set such that, byusing the brightness enhancement ratio t and a noise variance σ in theinput image, the attenuation F(t) becomes larger as the brightnessenhancement ratio t becomes larger, i.e., the attenuation F(t) becomessmaller as the brightness enhancement ratio t becomes smaller. Anexample of the calculation method for calculating the attenuation F(t)is illustrated below. In the following equation, k is a value manuallyset in advance.

${F( {t( {x,\lambda} )} )} = \{ {{\begin{matrix}{{\sigma (\lambda)} + {k \cdot ( {{t( {x,\lambda} )} - 1.0} )}} & {{{if}\mspace{14mu} {t( {x,\lambda} )}} \geq 1.0} \\{\sigma (\lambda)} & {else}\end{matrix}{or}{F( {t( {x,\lambda} )} )}} = \{ \begin{matrix}{k \cdot {\sigma (\lambda)} \cdot ( {{t( {x,\lambda} )} - 1.0} )} & {{{if}\mspace{14mu} {t( {x,\lambda} )}} \geq 1.0} \\{\sigma (\lambda)} & {else}\end{matrix} } $

According to an aspect of the second exemplary embodiment, the mist/fogcorrection processing for removing mist/fog from an image can suppress adrastic fluctuation on a low brightness side of the image to prevent ablack defect. According to another aspect of the second exemplaryembodiment, the mist/fog correction processing can prevent degradationof a contrast. Further, execution of the noise suppression processingbased on the correction factor enables to prevent a noise within theimage from being sharpened.

Still further, the image processing apparatus having a noise removalfunction together with a separation processing function for separating atexture and a region within the image from each other, in the imageanalysis, the separation processing function being relatively costly,the performance of the image processing apparatus can be enhanced aswell as the calculation cost thereof can be saved in comparison with acase where each function is performed simply in parallel.

A third exemplary embodiment of the present invention is describedbelow.

FIG. 3 is a block diagram illustrating an image processing apparatusaccording to the third exemplary embodiment.

The image processing apparatus according to the third exemplaryembodiment is configured to include, as illustrated in FIG. 3, askeleton component/texture component separation unit 30, a blackcorrection unit 31, an exposure correction unit 32, a white correctionunit 33, a mist/fog correction unit 34, a brightness restoration unit35, and a noise suppression unit 36.

The skeleton component/texture component separation unit 30 isconfigured to perform a skeleton component/texture component separationwith respect to a source image (i.e., an input image), wherein thesource image is separated into a skeleton component expressing color andbrightness of each region and a texture component and a noise component(hereinafter collectively referred to as texture component) as aresidual component of the source image. Further specifically, theskeleton component/texture component separation unit 30 is configured toperform a skeleton component/texture component separation using a totalvariation norm minimization wherein the brightness and color informationcan be extracted for each region. The total variation norm minimizationis a technique for removing a vibration component within an image as atexture component. According to the total variation norm minimization, askeleton component B of an image is extracted from a source image (i.e.,an input image) I by solving a minimization problem expressed by theequation (5). A combination with the multi-resolution analysis enablesnot only removal of a fine vibration component but also removal of a(low-frequency) vibration having a cycle to some extent.

The black correction unit 31 is configured to estimate a black referencelevel bl within the image by a histogram analysis with respect to thesource image (hereinafter referred to as input image) I and the skeletoncomponent B to perform a correction by using a compensation formulaexpressed by an equation (12). In the equation (12), T₁ indicates aninput image after being subjected to the black correction processing, B₁indicates a skeleton component after being subjected to the blackcorrection processing, x indicates a location of a pixel, and λindicates a color channel.

$\begin{matrix}{{T_{1}( {x,\lambda} )} = \{ {{\begin{matrix}0 & {{{if}\mspace{14mu} {I( {x,\lambda} )}} < {bl}} \\{\frac{1}{1 - {bl}}( {{I( {x,\lambda} )} - {bl}} )} & {else}\end{matrix}{B_{1}( {x,\lambda} )}} = \{ \begin{matrix}0 & {{{if}\mspace{14mu} {B( {x,\lambda} )}} < {bl}} \\{\frac{1}{1 - {bl}}( {{B( {x,\lambda} )} - {bl}} )} & {else}\end{matrix} } } & (12)\end{matrix}$

The above correction is required since an image with mist/fogsuperimposed thereon appears whitish in its entirety and the blackreference level is relatively high throughout the image. Therefore, itis preferred to detect the black reference level of the entire image andremove a range including no information to thereby widen a dynamic rangeas a whole. When estimating the black reference level bl, for example, ahistogram of luminance (i.e., brightness) as illustrated in FIG. 4 ismade for the entire image. Then, frequencies are added from a bottomside and luminance (i.e., brightness) reaching preliminary designatedcumulative frequency is set to the black reference level bl.Incidentally, the luminance (i.e., the brightness) reaching thepreliminary designated cumulative frequency may be multiplied by apredetermined ratio or may be provided with a limiter.

The exposure correction unit 32 is configured to calculate brightnessthroughout an image by a histogram analysis, calculate a gammacorrection parameter γ for changing brightness throughout the image totarget brightness, and execute gamma correction processing by using anequation (13) with respect to the black-corrected input image T₁ and theblack-corrected skeleton component B₁ for the purpose of adjustingbrightness throughout the image. Here, T₂ indicates an input image afterbeing subjected to the exposure correction processing and B₂ indicates askeleton component after being subjected to the exposure correctionprocessing.

T ₂(x,λ)=(T ₁(x,λ))^(γ)

B ₂(x,λ)=(B ₁(x,λ))^(γ)  (13)

The gamma correction parameter γ can be calculated, for example, by:

Γ=ln(q)/ln(p)

where an average value or a medium value of the luminance (i.e., thebrightness) throughout the image is p and target brightness is q. Here,ln is a logarithmic function.

The white correction unit 33 is configured to obtain environmental lightA(λ) by the image analysis and normalize a color of the environmentallight by using an equation (14) with respect to the exposure-correctedinput image T₂ and the exposure-corrected skeleton component B₂. Here,T₃ indicates the white-corrected input image and B₃ indicates thewhite-corrected skeleton component.

$\begin{matrix}{{{T_{3}( {x,\lambda} )} = {\frac{\underset{\forall\lambda}{\max \;}{A(\lambda)}}{A(\lambda)}{T_{2}( {x,\lambda} )}}}{{B_{3}( {x,\lambda} )} = {\frac{\underset{\forall\lambda}{\max \;}{A(\lambda)}}{A(\lambda)}{B_{2}( {x,\lambda} )}}}} & (14)\end{matrix}$

In the input image, a white balance is not always perfect but a contrastof colors throughout the image is widened due to the below mentionedmist/fog correction processing, which may invite a sharpened lost whitebalance. Consequently, it is desirable to preliminary normalize a colorof the environmental light.

The environmental light A(λ) may be calculated such that, for example,as illustrated in FIG. 5, a histogram of each color channel of the imageis made and frequencies are added from an upper side of the histogram toset a value reaching the preliminary designated cumulative frequency tothe environmental light A(λ). Incidentally, the value reaching thepreliminary designated cumulative frequency, i.e., A(λ), may bemultiplied by a predetermined ratio or may be provided with a limiter.

The mist/fog correction unit 34 is configured to perform a gammacorrection, while changing a gamma value for each pixel, from colorinformation of each pixel of a corrected input image T₃ obtained withreference to the skeleton component B₃. More specifically, with respectto a pixel at a location x, a value obtained such that a minimum valueof each color channel of the skeleton component B₃ is multiplied by theratio r is set to an amount of superimposition haze (0<haze<1) ofmist/fog of the image (see, an equation (15)). Then, a gamma value γ_(h)for correction is set by using an equation (16) and the correction isperformed by using an equation (17) based on the gamma value γ_(h).Here, T₄ indicates a mist/fog-removed input image and λ indicates acolor channel.

$\begin{matrix}{{haze} = {r \cdot {\min_{\forall\lambda}( {B_{3}( {x,\lambda} )} )}}} & (15) \\{\gamma_{h} = \frac{1.0}{1.0 - {haze}}} & (16) \\{{T_{4}( {x,\lambda} )} = ( {T_{3}( {x,\lambda} )} )^{\gamma_{h}}} & (17)\end{matrix}$

Similarly, with respect to a skeleton component B₃(x, λ), mist/fogcorrection processing is also applied to obtain B₄(x, λ).

$\begin{matrix}{{haze} = {r \cdot {\min_{\forall\lambda}( {B_{3}( {x,\lambda} )} )}}} & (15)^{\prime} \\{\gamma_{h} = \frac{1.0}{1.0 - {haze}}} & (16)^{\prime} \\{{B_{4}( {x,\lambda} )} = ( {B_{3}( {x,\lambda} )} )^{\gamma_{h}}} & (17)^{\prime}\end{matrix}$

Here, a reason why the gamma correction is used in the mist/fogcorrection processing is that, in the gamma correction (in a case of γvalue>1.0), a correction factor is relatively high on a high brightnessside of the input image and the correction factor is relatively low on alow brightness side of the input image. For example, as the amount ofsuperimposition haze of mist/fog becomes larger, the gamma value γ_(h)becomes larger according to the equations (16) and (16)′. Consequently,according to the equations (17) and (17)′, as the amount ofsuperimposition haze of mist/fog becomes larger, correction is performedso as to lower the brightness of the pixel. On the other hand, as theamount of superimposition haze of mist/fog becomes smaller, the gammavalue γ_(h) comes closer to 1 and the brightness of the pixel is lesscorrected.

The ratio r in the equations (15) and (15)′ may be changed according to,for example, the brightness lumi surrounding a target pixel.Incidentally, the ratio r has a positive correlation with a value of thebrightness lumi (i.e., When the brightness lumi increases, the ratio rbecomes higher, whereas, when the brightness lumi decreases, the ratio rbecomes lower.).

$r = \{ \begin{matrix}r_{\max} & {{{if}\mspace{14mu} {lumi}} > {th}} \\{r_{\max} \cdot {{lumi}/{th}}} & {else}\end{matrix} $

An example of a method for calculating the brightness lumi is shown asfollows.

lumi(x)=max_(∀λ)(B ₃(x,λ)

lumi(x)=(max_(∀λ)(B ₃(x,λ))+min_(∀λ)(B ₃(x,λ)))/2

The gamma correction is not the only example of the correction method ofthe mist/fog correction processing. For example, in a case where theamount of superimposition haze is calculated by using the ratio r thatis corrected by the brightness lumi, the following correction may beperformed.

${T_{4}( {x,\lambda} )} = \{ \begin{matrix}0 & {{{if}\mspace{14mu} {T_{3}( {x,\lambda} )}} < {haze}} \\{\frac{1.0}{1.0 - {haze}}( {{T_{3}( {x,\lambda} )} - {haze}} )} & {else}\end{matrix} $

The brightness restoration unit 35 is configured to restore brightnessthroughout the image whose brightness was lowered with respect to themist/fog removed input image T₄ and the mist/fog removed skeletoncomponent B₄. The mist/fog correction unit 34 comes to perform, in orderto suppress information loss caused by saturation, correction processingaround a saturation point 1.0. As a result, the brightness of theenvironmental light and the brightness throughout the image tend to bedark. The brightness throughout the image having been lowered isrestored by the brightness restoration unit 35. More specifically, themist/fog correction unit 34 is configured to calculate an index γ₂ forconverting the lowered brightness of the environmental light to targetbrightness and, by using the index γ₂, correct the brightness throughoutthe image according to an equation (18). Here, T₅ indicates abrightness-restored input image and B₅ indicates a brightness-restoredskeleton component.

T ₅(x,λ)=1.0−(1.0−T ₄(x,λ))^(γ) ²

B ₅(x,λ)=1.0−(1.0−B ₄(x,λ))^(γ) ²   (18)

The index γ₂ can be calculated by using the following equation:

γ₂=ln(1.0−q)/ln(1.0−p)

where a value of the mist/fog-corrected environmental light is p andtarget brightness is q. Incidentally, the value p may be a result of thewhite correction. Further, the target brightness q may be Max(A (λ))upon estimation of white correction.

Further, as seen from the following equation, a region where morecorrection is provided by the mist/fog correction unit 34 may be heavilyrestored.

T ₅′(x,λ)=ratio·T ₅(x,λ)+(1−ratio)·T ₄(x,λ)

Here, the ratio is determined by the amount of superimposition hazeaccording to the following equation.

${ratio} = \{ \begin{matrix}1.0 & {{{if}\mspace{14mu} {haze}} > {th}} \\{{haze}/{th}} & {else}\end{matrix} $

The noise suppression unit 36 is configured to suppress a sharpenednoise of an image. A local contrast is sharpened in the mist/fogcorrection processing, so that a noise as well as the texture componentof the image is sharpened. The noise suppression unit 36 is configuredto suppress the sharpened noise with respect to an input image T₅ and askeleton component B₅. More specifically, a brightness enhancement ratiot of the texture and the noise is calculated according to an equation(19) by using the skeleton component B₅ that has been subjected tocorrection processing similar to be provided on the input image.

$\begin{matrix}{{t( {x,\lambda} )} = {\frac{{T_{5}( {x,\lambda} )} - {B_{5}( {x,\lambda} )}}{{I( {x,\lambda} )} - {B( {x,\lambda} )}} = \frac{{sub}_{5}( {x,\lambda} )}{{sub}( {x,\lambda} )}}} & (19)\end{matrix}$

Then, based on the brightness enhancement ratio t, an attenuation F(t)of the texture component is set to generate an output image J accordingto an equation (20).

$\begin{matrix}{{J( {x,\lambda} )} = \{ \begin{matrix}{B_{5} + {{t( {x,\lambda} )} \cdot {{sign}( {{sub}( {x,\lambda} )} )} \cdot}} & {{{if}{{{sub}( {x,\lambda} )}}} > {F( {t( {x,\lambda} )} )}} \\( {{{{sub}( {x,\lambda} )}} - {F( {t( {x,\lambda} )} )}} ) & \; \\{B_{5}( {t( {x,\lambda} )} )} & {else}\end{matrix} } & (20)\end{matrix}$

Here, the attenuation F(t) of the texture component is set by using thebrightness enhancement ratio t and a noise variance σ in the input imagesuch that, as the brightness enhancement ratio t becomes larger, theattenuation F(t) becomes larger, i.e., as the brightness enhancementratio t becomes smaller, the attenuation F(t) becomes smaller. Anexemplary equation of a calculation method for calculating theattenuation F(t) is shown below, where k is a value manually set inadvance.

${F( {t( {x,\lambda} )} )} = \{ {{\begin{matrix}{{\sigma (\lambda)} + {k \cdot ( {{t( {x,\lambda} )} - 1.0} )}} & {{{if}\mspace{14mu} {t( {x,\lambda} )}} \geq 1.0} \\{\sigma (\lambda)} & {else}\end{matrix}{or}{F( {t( {x,\lambda} )} )}} = \{ \begin{matrix}{k \cdot {\sigma (\lambda)} \cdot ( {{t( {x,\lambda} )} - 1.0} )} & {{{if}\mspace{14mu} {t( {x,\lambda} )}} \geq 1.0} \\{\sigma (\lambda)} & {else}\end{matrix} } $

In the present method, the attenuation of the noise can be adaptivelyset based on the correction factor, so that less noise suppression isprovided to a region to which less amount of correction is made in themist/fog correction processing. Accordingly, an image can be outputwithout an original texture component being deleted.

FIG. 6 illustrates a source image (i.e., an input image) and an outputimage after being processed by the above described image processingapparatus. As seen from FIG. 6, whitishness and a lowered contrast ofthe image caused by mist/fog are restored and mist/fog is removed fromthe image. Also, the noise within the image is suppressed.

A fourth exemplary embodiment of the present invention is describedbelow.

The fourth exemplary embodiment has functions similar to those of thefirst exemplary embodiment except for the brightness restoration unit 3.An effect of mist/fog correction processing can be produced satisfactorywith respect to an image hardly affected by degradation of brightness.The fourth exemplary embodiment is described below.

FIG. 7 is a block diagram illustrating an image processing apparatusaccording to the fourth exemplary embodiment of the present invention.

The image processing apparatus of the fourth exemplary embodiment isconfigured to include a skeleton separation unit 40 and a mist/fogcorrection unit 41.

The skeleton separation unit 40 has function similar to those of theskeleton separation unit 1. That is, the skeleton separation unit 40 isconfigured to separate a skeleton component illustrating a perspectivestructure from a source image (i.e., an input image) input to the imageprocessing apparatus. An example of the separation method may include askeleton component/texture component separation method using a totalvariation norm minimization wherein the brightness and color informationcan be extracted for each region. The total variation norm minimizationis a technique for removing a vibration component within an image as atexture component or a noise component. In the total variation normminimization, removal of the texture component or the noise componentenables extraction of the skeleton component of the source image (i.e.,the input image).

More specifically, by solving a minimization problem expressed by theequation (5), a skeleton component B of an image is extracted from asource image (hereinafter referred to as input image) I. In combinationwith a multi-resolution analysis, not only removal of a fine vibrationcomponent but also removal of a (low-frequency) vibration having a widecycle to some extent can be achieved.

The mist/fog correction unit 41 has functions similar to those of themist/fog correction unit 2. That is, the mist/fog correction unit 41 isconfigured to calculate, based on color information in each colorchannel of the skeleton component, a correction factor that lowersbrightness of a pixel as the pixel becomes brighter and correct, basedon the correction factor, brightness of each pixel of an input image toremove mist/fog. In a correction of the brightness, execution of a localgamma correction suppresses a drastic fluctuation on a low brightnessside of the image, thereby preventing a black defect. More specifically,the mist/fog correction unit 41 is configured to calculate, from aminimum value of color channel of each pixel of the skeleton component,an amount of superimposition of mist/fog of the pixel and calculate,from the amount of superimposition of mist/fog, a gamma value (i.e., acorrection factor) for correction of a pixel to perform a gammacorrection by using thus calculated gamma value with respect to the eachpixel. Incidentally, the gamma value (i.e., the correction factor) maybe determined not only with color information in each color channel butalso additionally with brightness surrounding a target pixel.

More specifically, the mist/fog correction unit 41 performs the gammacorrection, with reference to the skeleton component B, while changingthe gamma value for each pixel based on the color information of eachpixel of an input image I. Further specifically, with respect to a pixelat a location x, a value obtained such that a minimum value of eachcolor channel of the skeleton component B is multiplied by a ratio r isset to an amount of superimposition haze (0<haze<1) of mist/fog of theimage (see, the equation (6)). Then, a gamma value γ_(h) is set for thepurpose of a correction by using the equation (7) and correction isperformed based on the gamma value γ_(h) according to the equation (8).

According to an aspect of the fourth exemplary embodiment, high qualitymist/fog correction processing can be performed even in a case where apixel whose value is 0 in either one of a red (R) channel, a green (G)channel, and a blue (B) channel does not exist around the target pixeland in a case where a different value between color bands is small.

In the above described exemplary embodiments, each unit is constitutedby hardware. However, a program for causing an information processingapparatus (e.g., a CPU) to perform the above described processing can beused instead of the hardware.

One part or an entirety of the above exemplary embodiment can beexpressed as the following notes, but the invention is not limitedthereto.

(Supplementary note 1) An image processing apparatus comprising:

a skeleton component separation unit configured to separate a skeletoncomponent illustrating a perspective structure from a source image; and

a mist/fog correction unit configured to calculate, based on colorinformation in each color channel of the skeleton component, acorrection factor and correct, based on the correction factor,brightness of each pixel of the source image.

(Supplementary note 2) The image processing apparatus according toSupplementary note 1, wherein the mist/fog correction unit is configuredto calculate, based on color information in each color channel of theskeleton component, a correction factor that lowers brightness of apixel as the pixel is brighter and correct, based on the correctionfactor, brightness of each pixel of the source image.

(Supplementary note 3) The image processing apparatus according toSupplementary note 1 or Supplementary note 2, wherein the mist/fogcorrection unit is configured to calculate, from a minimum value ofcolor channel of each pixel of the skeleton component, an amount ofsuperimposition of mist/fog of the each pixel and calculate a gammavalue for correction of the each pixel from the amount ofsuperimposition of mist/fog to perform a gamma correction by using thegamma value with respect to the each pixel.

(Supplementary note 4) The image processing apparatus according to anyone of Supplementary notes 1 through 3, further comprising:

a brightness restoration unit configured to restore brightness of thebrightness-corrected source image to brightness of environmental lightas target brightness.

(Supplementary note 5) The image processing apparatus according toSupplementary note 4, wherein the brightness restoration unit isconfigured to restore brightness more for a pixel whose brightness hasbeen corrected more by the mist/fog correction unit.

(Supplementary note 6) The image processing apparatus according toSupplementary note 4 or Supplementary note 5:

wherein the skeleton component separation unit is configured to separatea source image into a skeleton component illustrating a perspectivestructure and a texture component as a residual component of the sourceimage;

wherein the mist/fog correction unit is configured to calculate, basedon color information in each color channel of the skeleton component, acorrection factor that lowers brightness of a pixel as the pixel isbrighter and correct, based on the correction factor, brightness of eachpixel of the source image and each pixel of the skeleton component;

wherein the brightness restoration unit is configured to restorebrightness of the brightness-corrected source image and thebrightness-corrected skeleton component to brightness of environmentallight as target brightness; and

wherein the image processing apparatus further comprises a noisesuppression unit configured to suppress a noise of the texture componentbased on the brightness-restored source image, the brightness-restoredskeleton component, and the texture component and generate an outputimage based on the noise-suppressed texture component and thebrightness-restored skeleton component.

(Supplementary note 7) The image processing apparatus according toSupplementary note 6, wherein the noise suppression unit is configuredto calculate a brightness enhancement ratio based on a ratio between thesource image and the skeleton component and the brightness-restoredsource image and the brightness-restored skeleton component, attenuatethe texture component based on the brightness enhancement ratio, andsuppress, based on the attenuated texture component, a noise of thebrightness-restored skeleton component.

(Supplementary note 8) The image processing apparatus according to anyone of Supplementary notes 1 through 7, further comprising:

a black correction unit configured to detect a black reference levelthroughout an image and remove a range with no information to enlarge adynamic range with respect to the source image and the skeletoncomponent;

an exposure correction unit configured to calculate brightnessthroughout the image of the black-corrected source image and theblack-corrected skeleton component and correct the brightness throughoutthe image to target brightness; and

a white correction unit configured to obtain environmental light of theexposure-corrected source image and the exposure-corrected skeletoncomponent and normalize a color of the environmental light;

wherein the mist/fog correction unit is configured to make a correctionby using the source image and the skeleton component each after beingprocessed by the black correction unit, the exposure correction unit,and the white correction unit.

(Supplementary note 9) The image processing apparatus according to anyone of Supplementary notes 1 through 7, wherein the mist/fog correctionunit is configured to correct brightness of each pixel of the sourceimage by using a correction factor obtained from each pixel of theskeleton component.

(Supplementary note 10) An image processing method comprising:

separation processing configured to separate a skeleton componentillustrating a perspective structure from a source image; and

mist/fog correction processing configured to calculate a correctionfactor based on color information in each color channel of the skeletoncomponent and correct, based on the correction factor, brightness ofeach pixel of the source image.

(Supplementary note 11) The image processing method according toSupplementary note 10, wherein the mist/fog correction processing isconfigured to calculate, based on color information in each colorchannel of the skeleton component, a correction factor that lowersbrightness of a pixel as the pixel is brighter and correct, based on thecorrection factor, brightness of each pixel of the source image.

(Supplementary note 12) The image processing method according toSupplementary note 10 or Supplementary note 11, wherein the mist/fogremoval unit is configured to calculate, from a minimum value of colorchannel of each pixel of the skeleton component, an amount ofsuperimposition of mist/fog of the each pixel and calculate a gammavalue for correction of the each pixel from the amount ofsuperimposition of mist/fog to perform a gamma correction by using thegamma value with respect to the each pixel.

(Supplementary note 13) The image processing method according to any oneof Supplementary notes 10 through 12, further comprising a brightnessrestoration unit configured to restore brightness of thebrightness-corrected source image to brightness of environmental lightas target brightness.

(Supplementary note 14) The image processing method according toSupplementary note 13, wherein the brightness restoration processing isconfigured to restore brightness more for a pixel whose brightness hasbeen corrected more by the mist/fog correction unit.

(Supplementary note 15) The image processing method according toSupplementary note 13 or Supplementary note 14:

wherein the separation processing is configured to separate a sourceimage into a skeleton component illustrating a perspective structure anda texture component as a residual component of the source image;

wherein the mist/fog correction processing is configured to calculate,based on color information in each color channel of the skeletoncomponent, a correction factor that lowers brightness of a pixel as thepixel is brighter and correct, based on the correction factor,brightness of each pixel of the source image and each pixel of theskeleton component;

wherein the brightness restoration processing is configured to restorebrightness of the brightness-corrected source image and thebrightness-corrected skeleton component to brightness of environmentallight as target brightness; and

wherein the image processing method further comprises a noisesuppression processing configured to suppress a noise of the texturecomponent based on the brightness-restored source image and thebrightness-restored skeleton component and the texture component andgenerate an output image from the noise-suppressed texture component andthe brightness-restored skeleton component.

(Supplementary note 16) The image processing method according toSupplementary note 15, wherein the noise suppression processing isconfigured to calculate a brightness enhancement ratio based on a ratiobetween the source image and the skeleton component and thebrightness-restored source image and the brightness-restored skeletoncomponent, attenuate the texture component based on the brightnessenhancement ratio, and suppress, based on the attenuated texturecomponent, a noise of the brightness-restored skeleton component.

(Supplementary note 17) The image processing method according to any oneof Supplementary notes 10 through 16, further comprising:

black correction processing configured to detect a black reference levelthroughout the image and remove a range with no information to enlarge adynamic range with respect to the source image and the skeletoncomponent;

exposure correction processing configured to calculate brightnessthroughout the image of the black-corrected source image and theblack-corrected skeleton component to correct brightness throughout theimage to target brightness; and

white correction processing configured to obtain environmental light ofthe exposure-corrected source image and the exposure-corrected skeletoncomponent to normalize a color of the environmental light;

wherein the mist/fog correction processing is configured to performcorrection by using the source image and the skeleton component eachafter being subjected to the black correction processing, the exposurecorrection processing, and the white correction processing.

(Supplementary note 18) The image processing method according to any oneof Supplementary notes 9 through 17, wherein the mist/fog correctionprocessing is configured to correct brightness of each pixel of thesource image by using a correction factor obtained from each pixel ofthe skeleton component.

(Supplementary note 19) A program for causing a computer to perform:

separation processing configured to separate a skeleton componentillustrating a perspective structure from a source image; and

mist/fog correction processing configured to calculate a correctionfactor based on color information in each color channel of the skeletoncomponent and correct, based on the correction factor, brightness ofeach pixel of the source image.

(Supplementary note 20) The program according to Supplementary note 19,wherein mist/fog correction processing is configured to calculate, basedon color information in each color channel of the skeleton component, acorrection factor that lowers brightness of a pixel as the pixel isbrighter and correct, based on the correction factor, brightness of eachpixel of the source image.

(Supplementary note 21) The program according to Supplementary note 19or Supplementary note 20, wherein the mist/fog correction processing isconfigured to calculate, from a minimum value of color channel of eachpixel of the skeleton component, an amount of superimposition ofmist/fog of the each pixel and calculate, from the amount ofsuperimposition of the mist/fog, a gamma value for correction of theeach pixel to perform a gamma correction by using the gamma value withrespect to the each pixel.

(Supplementary note 22) The program according to any one ofSupplementary notes 19 through 21 for causing a computer to performbrightness restoration processing wherein the brightness of thebrightness-corrected source image is corrected to brightness ofenvironmental light as target brightness.

(Supplementary note 23) The program according to Supplementary note 22,wherein the brightness restoration processing is configured to restorethe brightness more for a pixel whose brightness has been corrected moreby the mist/fog correction unit.

(Supplementary note 24) The program according to Supplementary note 22or Supplementary note 23:

wherein the separation processing is configured to separate a sourceimage into a skeleton component illustrating a perspective structure anda texture component as a residual component of the source image;

wherein the mist/fog correction processing is configured to calculate,based on color information in each color channel of the skeletoncomponent, a correction factor that lowers brightness of a pixel as thepixel is brighter and correct, based on the correction factor,brightness of each pixel of the source image and each pixel of theskeleton component;

wherein the brightness restoration processing is configured to restorebrightness of the brightness-corrected source image and thebrightness-corrected skeleton component to brightness of environmentallight as target brightness; and

wherein the program causes a computer to perform noise suppressionprocessing configured to suppress a noise of the texture component basedon the brightness-restored source image, the brightness-restoredskeleton component, and the texture component and generate an outputimage from the noise-suppressed texture component and thebrightness-restored skeleton component.

(Supplementary note 25) The program according to Supplementary note 24,wherein the noise suppression processing is configured to calculate abrightness enhancement ratio based on a ratio between the source imageand the skeleton component and the brightness-restored source image andthe brightness-restored skeleton component and attenuate, based on thebrightness enhancement ratio, the texture component to suppress a noiseof the brightness-restored skeleton component based on the attenuatedtexture component.

(Supplementary note 26) The program according to any one ofSupplementary notes 19 through 25 for causing a computer to perform:

black correction processing configured to detect a black reference levelthroughout the image and remove a range with no information to enlarge adynamic range with respect to the source image and the skeletoncomponent;

exposure correction processing configured to calculate brightnessthroughout the image of the black-corrected source image and theblack-corrected skeleton component to correct brightness throughout theimage to target brightness; and

white correction processing configured to obtain environmental light ofthe exposure-corrected source image and the exposure-corrected skeletoncomponent to normalize a color of the environmental light;

wherein the mist/fog correction processing is configured to performcorrection by using the source image and the skeleton component eachafter being subjected to the black correction processing, the exposurecorrection processing, and the white correction processing.

(Supplementary note 27) The program according to any one ofSupplementary notes 19 through 26, wherein the mist/fog correctionprocessing is configured to correct brightness of each pixel of thesource image by using a correction factor obtained from each pixel ofthe skeleton component.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The present inventioncan be carried out so as not to depart from the spirit and scope of theinvention.

This application is based upon and claims the benefit of priority fromJapanese patent application No. 2012-249458, filed on Nov. 13, 2012, thedisclosure of which is incorporated herein in its entirety by reference.

REFERENCE SIGNS LIST

-   1 skeleton component separation unit-   2 mist/fog correction unit-   3 brightness restoration unit-   11 skeleton component/texture component separation unit-   12 mist/fog correction unit-   13 brightness restoration unit-   14 noise correction unit-   30 skeleton component/texture component separation unit-   31 black correction unit-   32 exposure correction unit-   33 white correction unit-   34 mist/fog correction unit-   35 brightness restoration unit-   36 noise correction unit-   40 skeleton component separation unit-   41 mist/fog correction unit

What is claimed is:
 1. An image processing apparatus comprising: askeleton component separation unit configured to separate a skeletoncomponent illustrating a perspective structure from a source image; anda mist/fog correction unit configured to calculate, based on colorinformation in each color channel of the skeleton component, acorrection factor and correct, based on the correction factor,brightness of each pixel of the source image.
 2. The image processingapparatus according to claim 1, wherein the mist/fog correction unit isconfigured to calculate, based on color information in each colorchannel of the skeleton component, a correction factor that lowersbrightness of a pixel as the pixel is brighter and correct, based on thecorrection factor, brightness of each pixel of the source image.
 3. Theimage processing apparatus according to claim 1, wherein the mist/fogcorrection unit is configured to calculate, from a minimum value ofcolor channel of each pixel of the skeleton component, an amount ofsuperimposition of mist/fog of the each pixel and calculate a gammavalue for correction of the each pixel from the amount ofsuperimposition of mist/fog to perform a gamma correction by using thegamma value with respect to the each pixel.
 4. The image processingapparatus according to claim 1, further comprising: a brightnessrestoration unit configured to restore brightness of thebrightness-corrected source image to brightness of environmental lightas target brightness.
 5. The image processing apparatus according toclaim 4, wherein the brightness restoration unit is configured torestore brightness more for a pixel whose brightness has been correctedmore by the mist/fog correction unit.
 6. The image processing apparatusaccording to claim 4: wherein the skeleton component separation unit isconfigured to separate a source image into a skeleton componentillustrating a perspective structure and a texture component as aresidual component of the source image; wherein the mist/fog correctionunit is configured to calculate, based on color information in eachcolor channel of the skeleton component, a correction factor that lowersbrightness of a pixel as the pixel is brighter and correct, based on thecorrection factor, brightness of each pixel of the source image and eachpixel of the skeleton component; wherein the brightness restoration unitis configured to restore brightness of the brightness-corrected sourceimage and the brightness-corrected skeleton component to brightness ofenvironmental light as target brightness; and wherein the imageprocessing apparatus further comprises a noise suppression unitconfigured to suppress a noise of the texture component based on thebrightness-restored source image, the brightness-restored skeletoncomponent, and the texture component and generate an output image basedon the noise-suppressed texture component and the brightness-restoredskeleton component.
 7. The image processing apparatus according to claim6, wherein the noise suppression unit is configured to calculate abrightness enhancement ratio based on a ratio between the source imageand the skeleton component and the brightness-restored source image andthe brightness-restored skeleton component, attenuate the texturecomponent based on the brightness enhancement ratio, and suppress, basedon the attenuated texture component, a noise of the brightness-restoredskeleton component.
 8. The image processing apparatus according to claim1, further comprising: a black correction unit configured to detect ablack reference level throughout an image and remove a range with noinformation to enlarge a dynamic range with respect to the source imageand the skeleton component; an exposure correction unit configured tocalculate brightness throughout the image of the black-corrected sourceimage and the black-corrected skeleton component and correct thebrightness throughout the image to target brightness; and a whitecorrection unit configured to obtain environmental light of theexposure-corrected source image and the exposure-corrected skeletoncomponent and normalize a color of the environmental light; wherein themist/fog correction unit is configured to make a correction by using thesource image and the skeleton component each after being processed bythe black correction unit, the exposure correction unit, and the whitecorrection unit.
 9. The image processing apparatus according to claim 1,wherein the mist/fog correction unit is configured to correct brightnessof each pixel of the source image by using a correction factor obtainedfrom each pixel of the skeleton component.
 10. An image processingmethod comprising: separation processing configured to separate askeleton component illustrating a perspective structure from a sourceimage; and mist/fog correction processing configured to calculate acorrection factor based on color information in each color channel ofthe skeleton component and correct, based on the correction factor,brightness of each pixel of the source image.
 11. The image processingmethod according to claim 10, wherein the mist/fog correction processingis configured to calculate, based on color information in each colorchannel of the skeleton component, a correction factor that lowersbrightness of a pixel as the pixel is brighter and correct, based on thecorrection factor, brightness of each pixel of the source image.
 12. Theimage processing method according to claim 10, wherein the mist/fogremoval unit is configured to calculate, from a minimum value of colorchannel of each pixel of the skeleton component, an amount ofsuperimposition of mist/fog of the each pixel and calculate a gammavalue for correction of the each pixel from the amount ofsuperimposition of mist/fog to perform a gamma correction by using thegamma value with respect to the each pixel.
 13. The image processingmethod according to claim 10, further comprising a brightnessrestoration unit configured to restore brightness of thebrightness-corrected source image to brightness of environmental lightas target brightness.
 14. The image processing method according to claim13, wherein the brightness restoration processing is configured torestore brightness more for a pixel whose brightness has been correctedmore by the mist/fog correction unit.
 15. The image processing methodaccording to claim 13: wherein the separation processing is configuredto separate a source image into a skeleton component illustrating aperspective structure and a texture component as a residual component ofthe source image; wherein the mist/fog correction processing isconfigured to calculate, based on color information in each colorchannel of the skeleton component, a correction factor that lowersbrightness of a pixel as the pixel is brighter and correct, based on thecorrection factor, brightness of each pixel of the source image and eachpixel of the skeleton component; wherein the brightness restorationprocessing is configured to restore brightness of thebrightness-corrected source image and the brightness-corrected skeletoncomponent to brightness of environmental light as target brightness; andwherein the image processing method further comprises a noisesuppression processing configured to suppress a noise of the texturecomponent based on the brightness-restored source image and thebrightness-restored skeleton component and the texture component andgenerate an output image from the noise-suppressed texture component andthe brightness-restored skeleton component.
 16. The image processingmethod according to claim 15, wherein the noise suppression processingis configured to calculate a brightness enhancement ratio based on aratio between the source image and the skeleton component and thebrightness-restored source image and the brightness-restored skeletoncomponent, attenuate the texture component based on the brightnessenhancement ratio, and suppress, based on the attenuated texturecomponent, a noise of the brightness-restored skeleton component. 17.The image processing method according to claim 10, further comprising:black correction processing configured to detect a black reference levelthroughout the image and remove a range with no information to enlarge adynamic range with respect to the source image and the skeletoncomponent; exposure correction processing configured to calculatebrightness throughout the image of the black-corrected source image andthe black-corrected skeleton component to correct brightness throughoutthe image to target brightness; and white correction processingconfigured to obtain environmental light of the exposure-correctedsource image and the exposure-corrected skeleton component to normalizea color of the environmental light; wherein the mist/fog correctionprocessing is configured to perform correction by using the source imageand the skeleton component each after being subjected to the blackcorrection processing, the exposure correction processing, and the whitecorrection processing.
 18. The image processing method according toclaim 9, wherein the mist/fog correction processing is configured tocorrect brightness of each pixel of the source image by using acorrection factor obtained from each pixel of the skeleton component.19. A non-transitory computer readable storage medium storing a programfor causing a computer to perform: separation processing configured toseparate a skeleton component illustrating a perspective structure froma source image; and mist/fog correction processing configured tocalculate a correction factor based on color information in each colorchannel of the skeleton component and correct, based on the correctionfactor, brightness of each pixel of the source image.
 20. Thenon-transitory computer readable storage medium storing a programaccording to claim 19, wherein mist/fog correction processing isconfigured to calculate, based on color information in each colorchannel of the skeleton component, a correction factor that lowersbrightness of a pixel as the pixel is brighter and correct, based on thecorrection factor, brightness of each pixel of the source image. 21-27.(canceled)